Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout #10488

Merged
merged 1 commit into from
Feb 16, 2021
Merged

Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout #10488

merged 1 commit into from
Feb 16, 2021

Conversation

priyawadhwa
Copy link

@priyawadhwa priyawadhwa commented Feb 16, 2021 •
edited
Loading

About every 20% of the time, the KVM_Containerd integration test times out and only ~160 tests are run instead of the expected ~210. This can be seen in #10424 with this gopogh

After searching through jenkins logs for KVM_containerd, I found that the test seems to be timing out in the middle of TestStartStop:

  • example 1
  • example 2 (/var/lib/jenkins/go/src/k8s.io/minikube/test/integration/start_stop_delete_test.go:89)

A successful TestStartStop seems to take 15 minutes (--- PASS: TestStartStop (902.41s)) so reducing the time limit on this test to 25 minutes should still provide more than enough time to run it.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 16, 2021
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 16, 2021
@priyawadhwa
Copy link
Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Feb 16, 2021
@priyawadhwa priyawadhwa changed the title Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout wip: Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout Feb 16, 2021
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 16, 2021
@kubernetes kubernetes deleted a comment from minikube-pr-bot Feb 16, 2021
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 70.9s 68.1s 70.7s
Average time for minikube: 69.9s

Times for Minikube (PR 10488): 69.7s 64.8s 65.9s
Average time for Minikube (PR 10488): 66.8s

Averages Time Per Log

+--------------------------------+----------+---------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 10488) |
+--------------------------------+----------+---------------------+
| * minikube v1.17.1 on Debian   | 0.0s     | 0.0s                |
| 9.11 (kvm/amd64)               |          |                     |
| * Using the kvm2 driver based  | 0.0s     | 0.0s                |
| on user configuration          |          |                     |
| * Starting control plane node  | 0.0s     | 0.0s                |
| minikube in cluster minikube   |          |                     |
| * Creating kvm2 VM (CPUs=2,    | 43.6s    | 41.7s               |
| Memory=3700MB, Disk=20000MB)   |          |                     |
| ...                            |          |                     |
| * Preparing Kubernetes v1.20.2 | 23.2s    | 9.0s                |
| on Docker 20.10.2 ...          |          |                     |
|   - Generating certificates    | 0.0s     | 2.4s                |
| and keys ...                   |          |                     |
|   - Booting up control plane   | 0.0s     | 10.1s               |
| ...                            |          |                     |
|   - Configuring RBAC rules ... | 0.7s     | 1.4s                |
| * Verifying Kubernetes         | 1.7s     | 1.7s                |
| components...                  |          |                     |
| * Enabled addons:              | 0.6s     | 0.6s                |
| storage-provisioner,           |          |                     |
| default-storageclass           |          |                     |
| * Done! kubectl is now         | 0.0s     | 0.0s                |
| configured to use "minikube"   |          |                     |
| cluster and "default"          |          |                     |
| namespace by default           |          |                     |
+--------------------------------+----------+---------------------+

docker Driver
Times for minikube: 26.1s 27.4s 26.1s
Average time for minikube: 26.5s

Times for Minikube (PR 10488): 25.5s 27.0s 28.1s
Average time for Minikube (PR 10488): 26.9s

Averages Time Per Log

+--------------------------------+----------+---------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 10488) |
+--------------------------------+----------+---------------------+
| * minikube v1.17.1 on Debian   | 0.2s     | 0.2s                |
| 9.11 (kvm/amd64)               |          |                     |
| * Using the docker driver      | 0.1s     | 0.1s                |
| based on user configuration    |          |                     |
| * Starting control plane node  | 0.1s     | 0.1s                |
| minikube in cluster minikube   |          |                     |
| * Creating docker container    | 9.9s     | 9.7s                |
| (CPUs=2, Memory=3700MB) ...    |          |                     |
| * Preparing Kubernetes v1.20.2 | 15.2s    | 15.6s               |
| on Docker 20.10.2 ...          |          |                     |
| * Verifying Kubernetes         | 1.0s     | 1.1s                |
| components...                  |          |                     |
| * Enabled addons:              | 0.1s     | 0.1s                |
| storage-provisioner,           |          |                     |
| default-storageclass           |          |                     |
| * Done! kubectl is now         | 0.0s     | 0.0s                |
| configured to use "minikube"   |          |                     |
| cluster and "default"          |          |                     |
| namespace by default           |          |                     |
+--------------------------------+----------+---------------------+

@priyawadhwa priyawadhwa changed the title wip: Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout Reduce time limit on TestStartStop to fix occasional KVM_containerd timeout Feb 16, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 16, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [medyagh,priyawadhwa]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@priyawadhwa priyawadhwa merged commit eaaa51c into kubernetes:master Feb 16, 2021
@priyawadhwa priyawadhwa deleted the start-stop-delete branch February 16, 2021 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@medyagh medyagh medyagh approved these changes

@prasadkatti prasadkatti Awaiting requested review from prasadkatti

@RA489 RA489 Awaiting requested review from RA489

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@priyawadhwa @minikube-pr-bot @k8s-ci-robot @medyagh